Video capture system control using virtual cameras for augmented reality

ABSTRACT

There is provided a system and method for integrating a virtual rendering system and a video capture system using flexible camera control to provide an augmented reality. There is provided a method for integrating a virtual rendering system and a video capture system for outputting a composite render to a display, the method comprising obtaining, from the virtual rendering system, a virtual camera configuration of a virtual camera in a virtual environment, programming the video capture system using the virtual camera configuration to correspondingly control a robotic camera in a real environment, capturing a video capture feed using the robotic camera, obtaining a virtually rendered feed using the virtual camera, rendering the composite render by processing the feeds, and outputting the composite render to the display.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates generally to digital video. Moreparticularly, the present invention relates to digital video rendering.

2. Background Art

Modern commodity PC hardware and videogame consoles are often equippedwith sufficient processing capability to enable high-resolutionreal-time three-dimensional graphics rendering. Even portable devicessuch as mobile phones and handheld gaming systems are often equippedwith scaled down real-time three-dimensional graphics support. Suchlow-cost commodity graphics processing hardware has enabled a widevariety of entertainment and productivity applications to supportenhanced visual presentations for greater user engagement and enjoyment.

In particular, real-time three-dimensional graphics rendering has founditself as a highly supportive role in live broadcast programming. Forexample, coverage of sports and other live events can be readilyenhanced with composite renderings using three-dimensional graphics foralternative or substitute object rendering, strategy simulations,information boxes, alternative viewpoints, and other effects. Althoughthree-dimensional analysis tools that allow for real-time modificationof live footage exist, they are limited to modifying existing videocamera footage using prior camera paths and viewpoints. As a result,viewer engagement is low since the scope of analysis is so limited.

Accordingly, there is a need to overcome the drawbacks and deficienciesin the art by providing a way to create composite renderings using livefootage with real-time three-dimensional graphics rendering for highviewer impact and engagement.

SUMMARY OF THE INVENTION

There are provided systems and methods for integrating a virtualrendering system and a video capture system using flexible cameracontrol to provide an augmented reality, substantially as shown inand/or described in connection with at least one of the figures, as setforth more completely in the claims.

BRIEF DESCRIPTION OF THE DRAWINGS

The features and advantages of the present invention will become morereadily apparent to those ordinarily skilled in the art after reviewingthe following detailed description and accompanying drawings, wherein:

FIG. 1 presents a system for integrating a virtual rendering system anda video capture system using flexible camera control to provide anaugmented reality, according to one embodiment of the present invention;

FIG. 2 presents a diagram of a robotic camera path configured to match avirtual camera path, according to one embodiment of the presentinvention;

FIG. 3 presents a diagram of a composite render being generated,according to one embodiment of the present invention; and

FIG. 4 shows a flowchart describing the steps, according to oneembodiment of the present invention, by a virtual rendering system and avideo capture system may be integrated for outputting a composite renderof an augmented reality to a display.

DETAILED DESCRIPTION OF THE INVENTION

The present application is directed to a system and method forintegrating a virtual rendering system and a video capture system usingflexible camera control to provide an augmented reality. The followingdescription contains specific information pertaining to theimplementation of the present invention. One skilled in the art willrecognize that the present invention may be implemented in a mannerdifferent from that specifically discussed in the present application.Moreover, some of the specific details of the invention are notdiscussed in order not to obscure the invention. The specific detailsnot described in the present application are within the knowledge of aperson of ordinary skill in the art. The drawings in the presentapplication and their accompanying detailed description are directed tomerely exemplary embodiments of the invention. To maintain brevity,other embodiments of the invention, which use the principles of thepresent invention, are not specifically described in the presentapplication and are not specifically illustrated by the presentdrawings.

FIG. 1 presents a system for integrating a virtual rendering system anda video capture system using flexible camera control to provide anaugmented reality, according to one embodiment of the present invention.Diagram 100 of FIG. 1 includes virtual rendering system 110, videocapture system 130, master controller 150, composite render 155, livebroadcast link 156, and display 157. Virtual rendering system 110includes rendering engine controller 111, auxiliary rendering engine112, slave rendering engines 113 a-113 b, and virtual environment 120.Virtual environment 120 includes virtual cameras 121 a-121 b. Virtualcamera 121 a includes data 122 a. Virtual camera 121 b includes data 122b. Video capture system 130 includes camera motion controller 131 andreal environment 140. Real environment 140 includes robotic cameras 141a-141 b. Robotic camera 141 a includes data 142 a. Robotic camera 141 bincludes data 142 b.

Rendering engine controller 111, auxiliary rendering engine 112, slaverendering engine 113 a, and slave rendering engine 113 b may eachexecute on several separate servers, each comprising standard commodityPC hardware or a videogame console system. Alternatively, the engines ofvirtual rendering system 110 may be consolidated into a single server,or distributed remotely across a network to minimize the amount ofnecessary on-site hardware. Rendering engine controller 111 maycoordinate control and data sharing between the different renderingsubsystems, as shown in FIG. 1. Auxiliary rendering engine 112 mayprovide static graphics overlays and other graphical effects that do notrequire input from virtual environment 120. Slave rendering engines 113a-113 b each control virtual cameras 121 a-121 b respectively to receivea virtually rendered feed of virtual environment 120.

Data 122 a-122 b describing the configuration of virtual cameras 121a-121 b within virtual environment 120 may each include, for example,position data such as three-dimensional coordinates, camera field ofview orientation data such as camera angle, focal length and focusdistance, movement data such as a motion path or velocity andacceleration, and camera characteristics such as lens parameters, camerasize, center of lens, and other camera modeling details.Three-dimensional coordinates between virtual environment 120 and realenvironment 140 may be defined using a common frame of reference, suchas setting a particular corner of a field or a particular landmark as acommon (0, 0, 0) coordinate. The motion path may then describe thechanging of the above data parameters with respect to time, such as thethree-dimensional coordinates with respect to time or the camera fieldof view with respect to time. Slave rendering engines 113 a-113 b maythen modify data 122 a-122 b respectively to control camera paths forrespective virtual cameras 121 a-121 b.

Although virtual rendering system 110 of FIG. 1 depicts only two slaverendering engines each controlling exactly one virtual camera,alternative embodiments may use any arbitrary number of slave renderingengines to control any arbitrary number of virtual cameras. Morespecifically, each slave rendering engine may control more than onevirtual camera. Similarly, although video capture system 130 of FIG. 1only depicts two robotic cameras, alternative embodiments may use anyarbitrary number of robotic cameras to be controlled by camera motioncontroller 131 of video capture system 130. In this manner, thecomposite rendering system shown in FIG. 1 can be scaled to as manycamera angles and viewpoints as desired, in either virtual environment120 or real environment 140.

Real environment 140 corresponds to an actual physical environmentrepresented by virtual environment 120. For example, real environment140 might comprise an indoor or outdoor sports field or stadium, a golfcourse, a natural environment, an urban environment, or any otherlocale. Although examples have so far focused on sports entertainmentapplications, other focuses such as educational or informationalapplications may also benefit from this use of augmented reality.Virtual environment 120 may then be created using manualthree-dimensional environmental modeling, automated photographic orvideo extrapolation, or some other manual or automated method.

Master controller 150 may direct virtual rendering system 110 to controlvirtual cameras 121 a-121 b according to particular parameters, and alsodirect video capture system 130 to control robotic cameras 141 a-141 busing the same parameters. The particular parameters of camera behaviormight be dictated by manual control, by tracking the motion of aparticular object or focusing on a particular scene in either virtualenvironment 120 or real environment 140, by replaying a previouslyrecorded pattern of movement or another predetermined path, or by usingsome other criteria. Tracked objects may include, for example, a ball ora participating player of a game such as a sports match, and may bevirtual or real. Once the virtual and robotic cameras are properlyconfigured by appropriately programming the motion paths of data 122a-122 b and 142 a-142 b, master controller 150 may then query virtualrendering system 110 for virtually rendered feeds and video capturesystem 130 for video capture feeds. Master controller 150 may then actas a rendering controller by combining the feeds smoothly using standardbroadcast key technology such as chroma key or key/fill to generatecomposite render 155, which includes real and virtual feed elementsarranged in a specific desired manner for broadcast over live broadcastlink 156 to display 157. Live broadcast link 156 may comprise, forexample, a satellite uplink to a television studio, from where thebroadcasted material is disseminated to the general public. Display 157may then represent, for example, a television of a viewer watching thebroadcast.

FIG. 2 presents a diagram of a robotic camera path configured to match avirtual camera path, according to one embodiment of the presentinvention. Diagram 200 of FIG. 2 includes virtual environment 220 andreal environment 240. Virtual environment 220 includes virtual camera221, virtual camera path 223, virtual object 225, and virtual objectpath 226. Real environment 240 includes robotic camera 241 and roboticcamera path 243. With regards to FIG. 2, it should be noted that virtualenvironment 220 corresponds to virtual environment 120 from FIG. 1 andthat real environment 240 corresponds to real environment 140 fromFIG. 1. Moreover, although FIG. 2 only depicts a single virtual cameraand a single robotic camera for simplicity, alternative embodiments mayuse multiple virtual cameras and multiple robotic cameras.

As previously discussed in FIG. 1, master controller 150 may directvideo capture system 130 to control robotic cameras similarly to virtualcameras in virtual rendering system 110. FIG. 2 shows an example of thismanner of control, where robotic camera 241 is programmed to follow themovements of virtual camera 221. For example, virtual camera 221 may beprogrammed to focus on the movement of virtual object 225 followingvirtual object path 226. Thus, virtual camera 221 may follow virtualcamera path 223, with camera orientation following virtual object path226 as indicated by the dotted arrows. Virtual camera path 223 may thenbe recorded and programmed into robotic camera 241 of real environment240, so that robotic camera 241 can follow robotic camera path 243mirroring virtual camera path 223. Robotic camera 241 may comprise, forexample, a gantry supported fly-by camera, a programmable motion controlcamera, or another camera system supporting programmable movement.

As shown in FIG. 2, the camera orientation of robotic camera 241 movesas if it were following virtual object path 226 within real environment240, even though there is no corresponding real object for virtualobject 225 in real environment 240. By using the system described abovein FIG. 1, robotic camera 241 can thus be synchronized to the cameramovements of virtual camera 221. Composite rendering of real and virtualenvironments, also known as “augmented reality”, is thus facilitated, asthe camera views in virtual environment 220 and real environment 240 canbe matched according to any desired virtual camera path, opening uplimitless possibilities for dynamic camerawork.

The example shown in FIG. 2 might be used, for example, to present adynamic panning camera view showing a hypothetical ball pass defined byvirtual object 225 following virtual object path 226, even though such aball pass never happened in real environment 240. Thus, a compositerender might show a real sports field background and real players in avideo feed captured from real environment 240, but with a virtual ballrendered in virtual environment 220 as defined by virtual object 225following virtual object path 226. Thus, the composite render canprovide a realistic camera fly-by with background elements from realenvironment 240 and a virtual ball rendered from virtual environment220. This may be used, for example, to present hypothetical plays andstrategy analysis in a realistic and engaging manner for high viewerimpact.

FIG. 3 presents a diagram of a composite render being generated,according to one embodiment of the present invention. Diagram 300 ofFIG. 3 includes virtual rendering system 310, virtually rendered feeds315 a-315 b, video capture system 330, video capture feeds 335 a-335 b,master controller 350, composite render 355, live broadcast link 356,and display 357. With regards to FIG. 3, it should be noted that virtualrendering system 310 corresponds to virtual rendering system 110 fromFIG. 1, that video capture system 330 corresponds to video capturesystem 130, that master controller 350 corresponds to master controller150, that composite render 355 corresponds to composite render 155, thatlive broadcast link 356 corresponds to live broadcast link 156, and thatdisplay 357 corresponds to display 157.

As shown in FIG. 3, virtual rendering system 310 provides mastercontroller 350 with virtually rendered feeds 315 a-315 b, while videocapture system 330 provides master controller 350 with video capturefeeds 335 a-335 b. For example, video capture feed 335 a mightcorrespond to a feed generated by robotic camera 241 in FIG. 2, whereasvirtually rendered feed 315 a might correspond to a feed generated byvirtual camera 221 in FIG. 2. Virtually rendered feed 315 b maycorrespond to a feed created by an overhead virtual camera providing abird's eye overview of virtual environment 220 from FIG. 2, whereasvideo capture feed 335 b may correspond to a feed created by an overheadrobotic camera providing a bird's eye overview of real environment 240from FIG. 2.

Master controller 350 may then combine virtually rendered feed 315 a andvideo capture feed 335 a for an augmented reality fly-by scene and alsocombine virtually rendered feed 315 b and video capture feed 335 b foran augmented reality bird's eye overview scene. As previously discussed,master controller 350 may use standard broadcast key technologies tocombine the different feeds smoothly so that the juxtaposition of realand virtual elements is visually unobtrusive. Master controller 350 maythen use these combined scenes in composite render 355 through variouspresentation methods such as split screen, cascaded or tiled frames,“picture-in-picture”, three-dimensional surfaces, and other formattedlayouts. Master controller 350 may then forward composite render 355over live broadcast link 356 for showing on display 357. Mastercontroller 350 may repeat the above process of generating compositerender 355 in a periodic manner, such as 24, 30, or 60 times per secondor higher in order to accommodate a desired video broadcastingframerate.

Although FIG. 3 only shows a single composite render 355, alternativeembodiments may use several composite renders. For example, mastercontroller 350 may generate multiple composite renders to providedifferent camera views for multiple broadcast channels, to customizebased on a target broadcast region or audience demographics, to focus ona particular team in a sports match, or to support any otherbroadcasting application that may require multiple concurrent videostreams. By adding additional slave rendering engines and roboticcameras, augmented reality rendering systems can be readily scaled andconfigured to support large-scale projects.

FIG. 4 shows a flowchart describing the steps, according to oneembodiment of the present invention, by a virtual rendering system and avideo capture system may be integrated for outputting a composite renderof an augmented reality to a display. Certain details and features havebeen left out of flowchart 400 that are apparent to a person of ordinaryskill in the art. For example, a step may comprise one or more substepsor may involve specialized equipment or materials, as known in the art.While steps 410 through 460 indicated in flowchart 400 are sufficient todescribe one embodiment of the present invention, other embodiments ofthe invention may utilize steps different from those shown in flowchart400.

Referring to step 410 of flowchart 400 in FIG. 4 and diagram 100 of FIG.1, step 410 of flowchart 400 comprises obtaining, from virtual renderingsystem 110, data 122 a of virtual camera 121 a in virtual environment120. As shown in FIG. 1, master controller 150 may query renderingengine controller 111 for a virtual camera configuration concerningvirtual camera 121 a. Rendering engine controller 111 may then determinethat slave rendering engine 113 a controls virtual camera 121 a, andcorrespondingly send a request to slave rendering engine 113 a toretrieve data 122 a. Data 122 a may then be retrieved by slave renderingengine 113 a, for relay back to master controller 150 via renderingengine controller 111. As previously described, data 122 a may containvarious information concerning the configuration of virtual camera 121 asuch as three-dimensional position and movement, camera focus and view,camera model parameters, and other details.

Referring to step 420 of flowchart 400 in FIG. 4 and diagram 100 of FIG.1, step 420 of flowchart 400 comprises programming video capture system130 using data 122 a obtained from step 410 to correspondingly controlrobotic camera 141 a in real environment 140. For example, mastercontroller 150 may instruct camera motion controller 131 to programvalues into data 142 a to match data 122 a as closely as possible. Aspreviously discussed, data 122 a may include three-dimensionalcoordinates and camera field of view with respect to time. Assumingvirtual camera 221 and robotic camera 241 correspond to virtual camera121 a and robotic camera 141 a, the result of setting data 142 a tomatch data 122 a may be manifested by robotic camera path 243 mimickingvirtual camera path 223, as shown in FIG. 2.

Referring to step 430 of flowchart 400 in FIG. 4, diagram 200 of FIG. 2,and diagram 300 of FIG. 3, step 430 of flowchart 400 comprisescapturing, from video capture system 330, video capture feed 335 a ofreal environment 240 using robotic camera 241. Since the motion path ofrobotic camera 241 was previously programmed in step 420, step 430results in master controller 350 receiving video capture feed 335 acomprising fly-by footage according to robotic camera path 243.

Referring to step 440 of flowchart 400 in FIG. 4, diagram 200 of FIG. 2,and diagram 300 of FIG. 3, step 440 of flowchart 400 comprisesobtaining, from virtual rendering system 310, virtually rendered feed315 a of virtual environment 220 using virtual camera 221. As previouslydiscussed, virtual camera path 223 may be defined in any number of ways,such as by manual control, object tracking, recorded motion replay, orpredetermined paths. As shown in FIG. 2, virtual camera path 223 isdefined as an arc with the camera field of view following virtual object225 as it progresses through virtual object path 226. Thus, mastercontroller 350 may receive virtually rendered feed 315 a comprisingfly-by footage according to virtual camera path 223, wherein the feedincludes a rendering of virtual object 225.

Referring to step 450 of flowchart 400 in FIG. 4 and diagram 300 of FIG.3, step 450 of flowchart 400 comprises rendering composite render 355 byprocessing video capture feed 335 a from step 430 and virtually renderedfeed 315 a from step 440. As previously discussed, master controller 350may accomplish step 450 using standard broadcast key technology such aschroma key or key/fill techniques to isolate and combine components fromeach feed to produce a result with a smooth visual juxtaposition of realand virtual elements.

Referring to step 460 of flowchart 400 in FIG. 4 and diagram 300 of FIG.3, step 460 of flowchart 400 comprises outputting composite render 355from step 450 to display 357. As shown in FIG. 3, master controller 350may send composite render 355 using live broadcast link 356, which mightcomprise a satellite uplink to a broadcast station for publicdissemination. Eventually, composite render 355 shows on display 357,which might comprise the television of a viewer at home.

While the above steps 410-460 have been described with respect to asingle virtual camera, a single robotic camera, and a single compositerender, steps 410-460 may also be repeated as necessary to supportmultiple virtual cameras, multiple robotic cameras, and multiplecomposite renders, as previously described. In this manner, thedescribed rendering system can be flexibly scaled to larger projects byincreasing the number of slave rendering systems and robotic cameras tohandle additional feeds in real-time.

In this manner, live events such as sports can be enhanced withhigh-impact augmented reality segments by leveraging the cost effectivereal-time graphical capabilities of modern commodity PC hardware andgame consoles. This can provide networks with a competitive advantage bydrawing in and retaining greater viewership by providing compellingaugmented reality content while requiring only minor additionalinfrastructure outlays over standard rendering systems. Since commodityhardware parts are used and numerous effective virtual rendering systemsand engines are available for licensing, expensive proprietary systemsand vendor lockout may be avoided, further reducing total cost ofownership.

From the above description of the invention it is manifest that varioustechniques can be used for implementing the concepts of the presentinvention without departing from its scope. Moreover, while theinvention has been described with specific reference to certainembodiments, a person of ordinary skills in the art would recognize thatchanges can be made in form and detail without departing from the spiritand the scope of the invention. As such, the described embodiments areto be considered in all respects as illustrative and not restrictive. Itshould also be understood that the invention is not limited to theparticular embodiments described herein, but is capable of manyrearrangements, modifications, and substitutions without departing fromthe scope of the invention.

What is claimed is:
 1. A method for integrating a virtual renderingsystem and a video capture system for outputting a composite render to adisplay, the method comprising: tracking a moving virtual object in thevirtual environment by a first virtual camera traveling on a firstvirtual camera path and a second virtual camera traveling on a secondvirtual path, wherein the first virtual camera path and the secondcamera path are defined by tracking a real object moving in a realenvironment; obtaining a first virtually rendered feed of the virtualenvironment, including the moving virtual object, from the first virtualcamera, and a second virtually rendered feed of the virtual environment,including the moving virtual object, from the second virtual camera:obtaining, from the virtual rendering system, a first virtual cameraposition data and a first virtual camera movement data of the firstvirtual camera while the first virtual camera is travelling on thevirtual camera path and tracking the moving virtual object in thevirtual environment, and a second virtual camera position data and asecond virtual camera movement data of the second virtual camera whilethe second virtual camera is travelling on the second virtual camerapath and tracking the moving virtual object in the virtual environment;programming the video capture system using the first virtual cameraposition data and the virtual camera movement data of the first virtualcamera to control a first robotic camera in the real environment, andusing the second virtual camera position data and the second virtualcamera movement data of the second virtual camera to control a secondrobotic camera in the real environment to travel on a real pathcorresponding to the first virtual camera path and the second virtualcamera path, respectively; capturing, from the video capture system, afirst video capture feed of the real environment using the first roboticcamera while travelling on the real path and a second video capture feedof the real environment using the second robotic camera while travellingon the real path; rendering the composite render by processing the firstvideo capture feed, the first virtually rendered feed, the second videocapture feed and the second virtually rendered feed, including themoving virtual object; and outputting the composite render to thedisplay by simultaneously displaying the first virtually rendered feedand the second virtually rendered feed, wherein the first virtuallyrendered feed corresponds to a birdseye view of the virtual environment,and the second virtually rendered feed corresponds to a flyby view ofthe virtual environment.
 2. The method of claim 1, wherein the firstvirtual camera movement data includes a first motion path of the firstvirtual camera in the virtual environment, and wherein the control ofthe first robotic camera uses the first motion path in the realenvironment.
 3. The method of claim 2, wherein the first motion pathincludes a three-dimensional position with respect to time.
 4. Themethod of claim 2, wherein the first motion path includes a cameraorientation or field of view with respect to time.
 5. The method ofclaim 2, wherein the first motion path tracks a path of the movingvirtual object.
 6. The method of claim 5, wherein the moving virtualobject comprises a virtual player of a virtual game.
 7. The method ofclaim 2, wherein the first motion path is based on a predetermined path.8. The method of claim 1 further comprising: obtaining, from the virtualrendering system, a first virtual camera field of view orientation dataof the first virtual camera while tracking the moving virtual object inthe virtual environment, and wherein the camera field of vieworientation data includes a focal length and a focus distance of thevirtual camera.
 9. The method of claim 1, wherein the first virtualcamera movement data includes a velocity of the first virtual camera.10. The method of claim 1, wherein the first virtual camera movementdata includes an acceleration of the first virtual camera.
 11. Arendering controller for outputting a composite render to a display, therendering device comprising: a processor configured to: track a movingvirtual object in the virtual environment by a first virtual cameratraveling on a first virtual camera path and a second virtual cameratraveling on a second virtual path, wherein the first virtual camerapath and the second camera path are defined by tracking a real objectmoving in a real environment; obtain a first virtually rendered feed ofthe virtual environment and a second virtually rendered feed of thevirtual environment, including the moving virtual object, from the firstvirtual camera and the second virtual camera, respectively; obtain, froma virtual rendering system, a first virtual camera position data and afirst virtual camera movement data of the first virtual camera while thefirst virtual camera is travelling on the virtual camera path andtracking the moving virtual object in the virtual environment and asecond virtual camera position data and a second virtual camera movementdata of the second virtual camera while the second virtual camera istravelling on the second virtual camera path and tracking the movingvirtual object in the virtual environment; program a video capturesystem using the first virtual camera position data and the virtualcamera movement data of the first virtual camera to control a firstrobotic camera in the real environment to travel on a real pathcorresponding to the virtual camera path, and using the second virtualcamera position data and the second virtual camera movement data of thesecond virtual camera to control a second robotic camera in the realenvironment to travel on a real path corresponding to the first virtualcamera path and the second virtual camera path, respectively; capture,from the video capture system, a first video capture feed of the realenvironment using the first robotic camera while travelling on the realpath, and a second video capture feed of the real environment using thesecond robotic camera while travelling on the real path; render thecomposite render by combining the first video capture feed, the firstvirtually rendered feed, the second video capture feed and the secondvirtually rendered feed, including the moving virtual object; and outputthe composite render to the display by simultaneously displaying thefirst virtually rendered feed and the second virtually rendered feed,wherein the first virtually rendered feed corresponds to a birdseye viewof the virtual environment, and the second virtually rendered feedcorresponds to a flyby view of the virtual environment.
 12. Therendering controller of claim 11, wherein the first virtual cameramovement data includes a first motion path of the first virtual camerain the virtual environment, and wherein the control of the first roboticcamera uses the first motion path in the real environment.
 13. Therendering controller of claim 12, wherein the first motion path includesa three-dimensional position with respect to time.
 14. The renderingcontroller of claim 12, wherein the first motion path includes a cameraorientation or field of view with respect to time.
 15. The renderingcontroller of claim 12, wherein the first motion path tracks a path ofthe moving virtual object.
 16. The rendering controller of claim 15,wherein the moving virtual object comprises a virtual player of avirtual game.
 17. The rendering controller of claim 12, wherein thefirst motion path is based on a predetermined path.
 18. The renderingcontroller of claim 11, wherein the processor is further configured to:obtain, from the virtual rendering system, a first virtual camera fieldof view orientation data of the first virtual camera while tracking themoving virtual object in the virtual environment, and wherein the camerafield of view orientation data includes a focal length and a focusdistance of the virtual camera.
 19. The rendering controller of claim11, wherein the first virtual camera movement data includes a velocityand an acceleration of the first virtual camera.